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Abstract 

We show how real-number codes can be used to compress correlated sources and establish a new 
framework for distributed lossy source coding, in which we quantize compressed sources instead of 
compressing quantized sources. This change in the order of binning and quantization blocks makes it 
possible to model correlation between continuous-valued sources more realistically and compensate for 
the quantization error when the sources are completely correlated. We focus on the asymmetric case, 
i.e., lossy source coding with side information at the decoder, also known as Wyner-Ziv coding. The 
encoding and decoding procedures are described in detail for discrete Fourier transform (DFT) codes, 
both for syndrome- and parity-based approaches. We also extend the parity-based approach to the case 
where the transmission channel is noisy and perform distributed joint source-channel coding in this 
context. The proposed system is well suited for low-delay communications. Furthermore, the mean- 
squared reconstruction error (MSE) is shown to be less than or close to the quantization error level, the 
ideal case in coding based on binary codes. 

Index Terms 

Wyner-Ziv coding, distributed source coding, joint source-channel coding, real-number codes, BCH- 
DFT codes, syndrome, parity, low-delay. 
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The distributed source coding (DSC) studies compression of statistically dependent sources which 
do not communicate with each other [TTJ . The Wyner-Ziv coding problem Q, a special case of lossy 
DSC, considers lossy data compression with side information at the decoder. The current approach to 
the DSC of continuous-valued sources is to first convert them to discrete-valued sources and then apply 
lossless (Slepian-Wolf) coder (3j-||5j. Similarly, a practical Wyner-Ziv encoder consists of a quantizer and 
Slepian-Wolf encoder. There are, hence, quantization and binning losses for the source coder. Despite this, 
rate-distortion theory promises that block codes of sufficiently large length are asymptotically optimal, and 
they can be seen as vector quantizers followed by fixed-length coders (61. Therefore, practical Slepian- 
Wolf coders have been realized using different binary channel codes, e.g., LDPC and turbo codes [7]-[9|. 
These codes, however, are out of the question if low delay is imposed on the system as they may introduce 
excessive delay when the desired probability of error is very low. 

In this paper, we establish a new framework for the Wyner-Ziv coding and distributed lossy source 
coding, in general. We propose to first compress the continuous-valued sources and then quantize them, 
as opposed to the conventional approach. The new framework is compared against the existing one in 
Fig. [T] It introduces the use of real-number codes (see, e.g., 1 10|-fl4|]), to represent correlated sources 
with fewer samples, in the real field. 

To do compression, we generate syndrome or parity samples of the input sequence using a real- 
number channel code [10], similar to what is done to compress a binary sequence of data using binary 
channel codes. Then, we quantize these syndrome or parity samples and transmit them. There are still 
coding (binning) and quantization losses; however, since coding is performed before quantization, error 
correction is in the real field, and the quantization error can be corrected when two sources are completely 
correlated over a block of code. A second and more important advantage of this approach is the fact 
that the correlation channel model can be more realistic, as it captures dependency between continuous- 
valued sources rather than quantized sources. In the conventional approach, it is implicitly assumed that 
quantization of correlated signals results in correlated sequences in the binary domain; this may not 
necessarily be precise due to the nonlinearity of the quantization operation. To avoid any loss due to the 
inaccuracy of correlation model, we exploit the correlation between the continuous- valued sources before 
quantization. 

The new approach is also capable of alleviating the quantization error. This is possible because coding 
precedes quantization. Specifically, we use the Bose-Chaudhuri-Hocquenghem (BCH) DFT codes for 
compression |10J, p3|-p9|. Owing to DFT codes, the loss due to quantization can be decreased by 
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Fig. 1. The Wyner-Ziv coding based on binary and real-number codes. Both schemes can be simply extended to distributed 
source coding. 



a factor of ^ for an (n, k) code |19|-[21|. Additionally, reconstruction loss becomes zero if the two 
sources are perfectly correlated over one codeword. This is achieved in view of modeling the correlation 
between the two sources in the continuous domain. DFT codes also can exploit the temporal correlation 
typically found in many sources. 

Finally, the proposed scheme is more suitable for low-delay communications since, even by using 
short DFT codes, a reconstruction error less than the quantization error is attainable. Moreover, as another 
contribution of this paper, we use a single DFT code both to compress and protect sources against channel 
variations. This extends the Wyner-Ziv coding to the case when errors occur during transmission and 
proposes joint source-channel coding (JSCC) with side information at the decoder, within the realm of 
real-number codes. This scheme directly maps short source blocks into channel blocks, and thus it is 
well suited to low-delay coding. While the MSE performance of distributed JSCC systems with binary 
codes is limited to the quantization error level, the proposed scheme breaks through this limit. 

The paper is organized as follows. In Section [ill we motivate the new framework for lossy DSC. 



In Section III we study the encoding and decoding of DFT codes, and we adapt the subspace error 
localization to the Slepian-Wolf coder. Then in Section [W] we present the encoder and decoder for the 
Wyner-Ziv coding based on DFT codes, both for the syndrome and parity approaches. The system is 
extended to the noisy channel case in Section [V] Section [VI] presents the simulation results. Section VII 
provides our concluding remarks. 
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For notation, we use upper-case and boldface lower-case letters for vectors, boldface upper-case letters 
for matrices, (.) T for transpose, (.) H for conjugate transpose, (.)* for conjugate, and tr(.) for the trace. 
The dimensions of matrices are indicated by one and two subscripts, when required. 

II. Real-Number Codes for DSC 

A. Motivations 

Similar to error correction in finite fields, the basic idea of error correcting codes in the real field is to 
insert redundancy to a message vector to convert it to a longer vector, called the codeword. Nevertheless, 



the insertion of redundancy is done in the real field, i.e., before quantization and entropy coding [10], 
|TT[ . One main advantage of soft redundancy (real field codes) over hard redundancy (binary field codes) 
is that by using soft redundancy one can go beyond the quantization error level and thus reconstruct 
continuous-valued signals more accurately. 

We introduce the use of real-number codes in lossy compression of correlated signals. The proposed 
system is depicted in Fig. [T] Although it consists of the same blocks as the existing Wyner-Ziv coding 
scheme J3j, j4), the order of these blocks is changed here. Therefore binning is performed before 



quantization; we use DFT codes for this purpose flO| ], fl6| . This change in the order of binning and 
quantization blocks brings some advantages to lossy DSC, as described in the following: 

1) Realistic correlation model: In the existing framework for lossy DSC, correlation between two 
sources is modeled after quantization, i.e., in the binary domain. More precisely, correlation between 
quantized sources is usually modeled as a binary symmetric channel (BSC), mostly with known crossover 
probability (3J-J5J, |9j. Admittedly though, due to nonlinearity of the quantization operation, correlation 
between the quantized signals is not known accurately even if it is known in the continuous domain. 
This motivates us to investigate a method that exploits correlation between continuous- valued sources to 
perform DSC. 

2) Alleviating the quantization error: In lossy data compression with side information at the decoder, 
soft redundancy, added by DFT codes, can be used to correct both quantization errors and (correlation) 
channel errors. Thus, the loss due to quantization can be recovered, at least partly if not wholly. In fact, 
if the two sources are exactly the same over a codeword, the quantization error can be compensated 
for. That is, perfect reconstruction is attainable over the corresponding samples. The loss due to the 
quantization error is decreased by a factor of code rate even if correlation is not perfect, i.e., when 
(correlation) channel errors exist. This is because DFT codes are tight frames; hence, they minimize the 
MSE |il9l-||22l. 
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3) Low-delay communications: If communication is subject to low-delay constraints, we cannot use 
turbo or LDPC codes, as their performance is not satisfactory for short code length, low-delay coding 
has recently drawn a lot of attention; it is done by mapping short source blocks into channel blocks 
(23j, p4| . Whether low-delay requirement exists or not depends on the specific applications. However, 
even in the applications that low-delay transmission is not imperative, it is sometimes useful to consider 
low-dimensional systems for their low computational complexity. 

B. Correlation Channel Model 

Accurate modeling of the correlation between the sources plays a crucial role in the efficiency of 
the DSC systems. Most of the previous works assume that the correlation between continuous-valued 
sources can be modeled by an i.i.d. BSC in the binary field. This is not, however, exact because of 
the nonlinearity of quantization operation. Besides, independent errors in the sample domain are not 
translated to independent errors in the bit domain [25]. These issues can be avoided if we exploit the 
correlation between the continuous-valued sources before quantization. 

The correlation between the analog sources X and Y, in general, can be defined by 

Y = X + E, (1) 

where E is a real-valued random variable. Particularly, the above model represents some well-known 
models motivated in video coding and sensor networks. Let 

M(0,a 2 e ) w.p. qi, 

E~ <jV(0,of + cr?) w.p. q 2 , (2) 

w.p. l-gi-^2, 

V 

in which af 3> a\ and q\ + q 2 < 1. Then, for q\ = 1 or q 2 = 1 the Gaussian correlation is obtained, 
which is broadly used in the sensor networks literature. Further, for q\ + q 2 = 1 the Gaussian-Bernoulli- 
Gaussian (GBG) and for q\ + q 2 < 1, q\q 2 = the Gaussian-Erasure (GE) models are realized. The latter 
two models are more suitable for video applications p6*J . 

III. BCH-DFT CODES: Construction and Decoding 

In this section, we study a class of real-number codes that are employed for binning throughout this 
paper, investigate some properties of their syndrome, and adapt their decoding algorithm to the Slepian- 
Wolf coding. These codes are a family of Bose-Chaudhuri-Hocquenghem (BCH) codes in the real field 
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whose parity-check matrix H and Generator matrix G are defined based on the DFT matrix; they are, 
thus, known as BCH-DFT codes, or simply DFT codes. 



BCH-DFT codes |10| are linear block codes over the real or complex fields. Similar to other BCH 
codes, the spectrum of any codeword is zero in a block of d = n — k cyclically adjacent components, 
where d + 1 is the designed distance of that code [15]. The error correction capability of the code is, 
hence, given by t = |_f J - In this paper, we only consider real BCH-DFT codes, i.e., the BCH-DFT codes 
whose generator matrix has real entries only. 



A. Encoding 

An (n, k) real BCH-DFT code is defined by its generator and parity-check matrices. The generator 
matrix is given by 

G= J-W,:'£W,. (3) 



in which Wk and Wj^ respectively are the DFT and IDFT matrices of size k and n, and S is an n x 
matrix defined as 

//. o\ 



o h J 

where a = [~|] — L^ir^jfj] (3 = k — a, and the sizes of zero blocks are such that £ is an n x k matrix 
p6l~p8l, (27j- Then, for any u, this enforces the spectrum of the codeword 



V 



(4) 



c = Gu, 



(5) 



to have n — k consecutive zeros, which is required for any BCH code 1 15 1. The parity-check matrix H, 
on the other hand, is constructed by using the n — k columns of W„ corresponding to the n — k zero 
rows of Therefore, by virtue of the unitary property of , H is the null space of G, i.e., 



HG = 0. 

In the rest of this paper, we use the term DFT code in lieu of real BCH-DFT code. 



(6) 



'Knowing that n and k cannot be simultaneously even for a real DFT code |10| , 



show that a = \^]. 
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B. Decoding 

Before introducing the decoding algorithm, we define some notation and basic concepts. Let r = c + e 
be the received vector (a noisy version of c), where c is a codeword generated by ([5]). Suppose that e is 
an error vector with v nonzero elements at positions i\, . . . ,i u ; the magnitude of error at position i p is 
ei p . Then, we can compute 



s = Hr = H(c + e) = He, 



where s = [s\, S2, ■ ■ • , Sd] T is a complex vector with 



hit*.*? 



l+m 



m = 1, . . . , d 



(7) 



(8) 



P =i 



in which a = f^p] as defined in Q, X p = e « " , and p = 1, . . . , v. Next, we define the syndrome 
matrix 

51 S2 ... 

52 S3 ... Srf_ m+ 2 



Sm+1 



(9) 



forz/ + l<m<(i — z/ + l [16|. Also, we define the covariance matrix as 



(10) 



For decoding, we use the extension of the well-known Peterson-Gorenstein-Zierler (PGZ) algorithm to 
the real field [15]. This algorithm, aimed at detecting, localizing, and calculating the errors, works based 
on the syndrome of error. We summarize the main steps of this algorithm, adapted for a DFT code of 
length n, in the following. 

• Error detection: Determine the number of errors v by constructing a syndrome matrix and finding 
its rank. 

• Error localization: Find the coefficients Ai,...,A y of the error-locating polynomial A(x) = 
nj=i(l — xXi) whose roots Xf 1 , . . . ,X~ l are used to determine error locations; the errors are 
then in the locations i\, . . . ,i u such that X\ = u/ 1 , . . . , X v = uj 1u and ui = e~ J ~ . 

• Error calculation: Finally, calculate the error magnitudes by solving a set of linear equations whose 
constants coefficients are powers of Xj. 

In practice however, the received vector is distorted because of quantization. Let c and q denote the 
quantized codeword and quantization noise so that c = c + q. Therefore, r = c + e and its syndrome is 
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no longer equal to the syndrome of error because 

Hr = H(c + q + e) = s q + s = s, (11) 

where s q = Hq and q = [qi, q2, . . ■ , q n ] T is the quantization error. The distorted syndrome samples can 
be written as 

v n 

» p=i » p'=i 

The distorted syndrome matrix S 1 ™ and the corresponding covariance matrix R = S m S^ are defined 
similar to Q and ( [T0| ) but for the distorted syndrome samples. 

While the exact value of the error is determined neglecting quantization, the decoding becomes an 
estimation problem in the presence of quantization. Then, it is imperative to modify the PGZ algorithm 



to decode the errors reliably fl5j-|[19|]. In the remainder of this section, we discuss this problem and also 
improve the error detection and localization, by introducing a slightly different version of the existing 
methods. 

1) Error detection: For a given DFT code, we first fix an empirical threshold 9 based on eigende- 
composition of R when the codewords are error- free, i.e., when only the quantization error exist. This 
threshold is on the magnitude of eigenvalues, rather than the determinant of R. Let A max denote the 
largest eigenvalue of R for m = t+ 1. We find 9 such that, for a desired probability of correct detection 

Pd, 

Pr(A max < 9) > Pd . (13) 
In practice, when errors can occur, we estimate the number of errors by the number of eigenvalues of R 



greater than 9. This one step estimation is better than the original estimation in the PGZ algorithm [15], 



1 18 1, where the last row and column of St are removed until coming up with a non-singular matrix. The 
improvement comes from incorporating all syndrome samples, rather than some of them, for decision 
making. 

2) Error localization: The Subspace or coding-theoretic error localizations fT6} can be used to find 
the coefficients Ai, A v of the error-locating polynomial. The subspace approach is, however, more 
general than the coding-theoretic approach in the sense that it can use up to t + 1 — v degrees of freedom 
to localize v errors, compared to just one degree of freedom in the coding-theoretic approach. Hence, 
it improves the error localization in the presence of the quantization noise. We apply the subspace error 



localization both to the syndrome- and parity -based DSC (see Section IV I, similar to that in channel 
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coding 1 16 1 . To this end, we eigen-decompose the covariance matrix R = S m S^ for m = t + 10 It 
results in two orthogonal subspaces, the error and noise subspaces. There are m — v vectors in the noise 



subspace; we use them to localize errors using the MUSIC algorithm |16|, [28]. However, one should 
note that the way we compute the syndrome in DSC is different from that in channel coding which was 
presented in ([7]) and ( pT) ; this will be elaborated in Section IV 



3) Error calculation: This last step is rather simple. Let H e be the matrix consisting of the columns 
of H corresponding to error indices. The errors magnitude e = [e^, ej 2 , . . . , e^] T can be determined by 
solving 

H e e = s, (14) 

in a least squares sense, for example. This completes the error correction algorithm by calculating the 
error vector. 

C. Performance Compared to Binary Codes 

DFT codes by construction are capable of decreasing the quantization error. When there is no error, 
an (n, k) DFT code brings down the MSE below the quantization error level with a factor of R c = k/n 
|20j , pT| . This is also shown to be valid for channel errors, as long as channel can be modeled as an 
additive noise. To appreciate this, one can consider the generator matrix of a DFT code as analysis frame 



operator of a tight frame [21 1; it is known that frames are resilient to any additive noise, and tight frames 



reduce the MSE k/n times [29]. Hence, DFT codes can result in a MSE even less than the quantization 



error level whereas the MSE in a binary code is obviously lower-bounded by the quantization error level. 

IV. Wyner-Ziv Coding Using DFT Codes 

In this section, we use DFT codes to do Wyner-Ziv coding in the real field. This is accomplished 
by using DFT codes for binning and transmitting compressed signal, in the form of syndrome or parity 
samples, in a digital communication system. Let a; be a sequence of real random variables x\X2 ■ ■ ■ x n , 
and y be a noisy version of x such that yi = Xi + ei, where e« is continuous, i.i.d., and independent of 
Xi, as described in ([2]). The lower-case letters x, y, and e, respectively, are used to show the realization 
of the random variables X, Y, and E. Since e is continuous, this model precisely captures any variation 
of x, so it can model correlation between x and y accurately. This correlation model is important, for 

2 Although u + l<m<d— u + 1, the best result is achieved for m = t + 1 jl6^ For this m, the size of S m is either 

(t + 1) X (t + 1) or (t + 1) X t. 
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Fig. 2. The Wyner-Ziv coding using DFT codes: Syndrome approach. 



example, in video coders that exploit Wyner-Ziv concepts, e.g., when the decoder builds side information 
via extrapolation of previously decoded frames or interpolation of key frames [26]. In this paper, we use 
the GE model with q?, = in where e is a sparse vector inserting up to t errors in each codeword. 

A. Syndrome Approach 

1) Encoding: Given H, to compress an arbitrary sequence of data samples, we multiply it with H 
to find the corresponding syndrome samples s x = Hx. The syndrome is then quantized (s x = s x + q) 
and transmitted over a noiseless digital communication system, as shown in Fig. [2] Note that s x , s x are 
both complex vectors of length n — k. Thus, it seems that to transmit each sample we need to send two 
real numbers, one for the real part and one for the imaginary part, which halves the compression ratio. 
However, we observe that the syndrome of a DFT code is symmetric, as stated below. 

Lemma 1. The syndrome of an (n,k) DFT code satisfies 

s*d-m+n if k is odd, 



d—mi 



(15) 
if k is even, 



for m = 1, . . . , d and d = n — k. 



Proof: See Appendix VIII-A 



The above lemma implies that, for any d, it suffices to know the first [|] syndrome samples. We know 
that syndromes are complex numbers in general; however, from s^-m+i = it is clear that if d is 
an odd number, s m is real for m = [|]. Therefore, for an (n,k) code with odd k, transmitting n — k 
real numbers suffices. This results in a compression ratio of for the Slepian-Wolf encoder. Yet, one 



January 3, 2013 



DRAFT 



11 



can check that for even k, we have to transmit n — k + 1 real samples, which incurs a slight loss in 
compression. It is however negligible for large n. 

2) Decoding: The decoder estimates the input sequence from the received syndrome and side infor- 
mation y. To this end, it needs to evaluate the syndrome of (correlation) channel errors. This can be 
simply done by subtracting the received syndrome from the syndrome of the side information. Then, 
neglecting the quantization error, we obtain, 

s e = s y- s x, (16) 



and s e can be used to precisely estimate the error vector, as described in Section III-B In practice, 
however, the decoder knows s x = s x + q rather than s x . Therefore, only a distorted syndrome of error 
is available, i.e., 

s e = s y - s x = s e - q. (17) 



Hence, using the PGZ algorithm, error correction is accomplished based on ( fT7[ ). Note that, having 
computed the syndrome of error, decoding algorithm in a DSC using DFT codes is exactly the same 
as that in the channel coding problem. This is different from DSC techniques in the binary field which 
usually require a slight modification in the corresponding channel coding algorithm to be customized for 
DSC. 

B. Parity Approach 

The syndrome -based Wyner-Ziv coding is straightforward, but it is not clear how we can use it for 
noisy transmission. In the sequel, we explore an alternative approached, namely parity-based approach, 
to the Wyner-Ziv coding. 

1) Encoding: To compress x, the encoder generates the corresponding parity sequence p with n — k 
samples. The parity is then quantized and transmitted, as shown in Fig. [3j instead of transmitting the 
input data. To this end, we need to find a systematic generator matrix G sys , as G in ^ is not in the 
systematic form. 



A first approach is to find H sys and build G sys based on that [30]. Another, simpler, way is to obtain a 
systematic generator matrix directly from G. Let Gf. be a square matrix of size k composed of arbitrary 
rows of G. We see that Gk is invertible because using ([3]) any k x k submatrix of G can be represented 
as product of a Vandermonde matrix and the DFT matrix This is also proven using a different 



approach in |21 1, where it is shown that any subframe of G is a frame, and its rank is equal to k. Hence, 
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Fig. 3. The Wyner-Ziv coding using DFT codes: Parity approach. 



a systematic generator matrix is given by 

G sys = GG k l . (18) 

Besides, from HG = 0, it is clear that 

HG sys = 0. (19) 

Therefore, we do not need to calculate H sys , and the same parity-check matrix H can be used for 
decoding in the parity approach. It is also obvious that G sys is a real matrix. The question that remains 
to be answered is whether G sys corresponds to a BCH code. To generate a BCH code, G sys must have n—k 
consecutive zeros in the transform domain. The Fourier transform of this matrix W n G sys = (W n G)G^ 1 
satisfies the required condition because W n G, the Fourier transform of original matrix, satisfies that. 



It should be emphasized that one can arbitrarily choose the rows of Gt in ( [18] ); this results in (^j 
systematic generator matrix for an (n, k) DFT code. Although any of those systematic codes can be used 
for encoding, the dynamic range of the generated parity samples depends on their relative position of the 



chosen rows (27J. In pi] Theorem 7], we have proved that when using these systematic frames for error 
correction, the mean-squared reconstruction error is minimized when the systematic rows are chosen as 
evenly spaced as possible. In the extreme scenario, where the systematic rows are equally spaced, the 
systematic frame is also tight. This is realized only when n is an integer multiple of k. Such a frame 



lends itself well to minimize reconstruction error [20]-[22], [32]. 

Finally, seeing that parity samples are real numbers, using an (n, k) DFT code, a compression ratio of 
-^r is achieved. Obviously, a compression ratio of is achievable if we use a (2n — k, n) DFT code. 

2) Decoding: A parity decoder estimates the input sequence from the received parity and side infor- 
mation y. Similar to the syndrome approach, at the decoder, we need to find the syndrome of (correlation) 
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channel errors. To do so, we append the parity to the side information and form a vector of length n 
whose syndrome, neglecting quantization, is equal to the syndrome of error. That is, 





y 




X 




e 


z = 


p 




p 


+ 






(20) 



and e' = [e | 0] T . Hence, 



s, = s 



e' ■ 



Similarly, when quantization is involved (p = p + q), we get 

y o 

z = = z + = G sys x + e' + q', 

P J [q _ 

and 

Sz = S e > -\- Sq' , 



(21) 



(22) 



(23) 



where, q' = [q | 0] T , and s q > = Hq' . Therefore, we obtain a distorted version of error syndrome. In 
both cases, the rest of the algorithm, which is based on the syndrome of error, is similar to that in the 



channel coding problem using DFT codes, as explained in Section III-B2 



Error localization algorithm for the parity-based DSC 1 30 1 can be further improved using the fact 
that parity samples are error-free. As parity samples are transmitted over a noiseless channel, the error 
locations, in the codewords, are restricted to the systematic samples. Therefore, we can exclude the set 
of roots corresponding to the location of the parity samples. We call this adapted error localization. 
Furthermore, it makes sense to use a code with evenly-spaced parity samples so as to optimize the 
location of error-free and error-prone samples in the codewords. Such a code maximizes the distance 
between the error-prone roots of the code; hence, it helps decrease the probability of incorrect decision. 



C. Comparison Between the Two Approaches 

1 ) Rate: As it was shown earlier, using an (n, k) code the compression ratio in the syndrome and parity 
approaches, respectively, is and ^3^. Hence, for a given code, the parity approach is - = R c < 1 
times less efficient than the syndrome approach. Conversely, we can find two different codes that result 
in same compression ratio, say We mentioned that in the parity approach, a (2n — k, n) code can 
be used for this matter, whereas an (n, k) DFT code gives the desired compression ratio in the syndrome 
approach. Thus, for a given compression ratio the syndrome approach implies a code with smaller rate 
compared to the code required in the parity approach. 
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2) Performance: From frame theory, we know that DFT frames are tight, and an (n, k) tight frame 



reduces the quantization error with a factor of R c = - |20|, |21 1, [29|. This result is extended to errors, 



given that channel can be modeled by an additive noise [19]. The MSE performance of systematic DFT 



frames also linearly depends on the code rate, though they are not necessarily tight [27 1, 1 3 1 1. Therefore, 
for codes with the same error correction capability, the lower the code rate the better the error correction 
performance. This implies a better performance for syndrome-based DSC. Further, a (2n — k, n) code 
has n — k roots more than an (n, k) code on the unit circle; hence, the roots are closer to each other and 
the probability of incorrect localization of errors increases. 

Additionally, from rate-distortion theory we know that the rate required to transmit a Gaussian source 



logarithmically increases with the source variance [33 1. Thus, in a system that uses a real-number code 



for encoding, since coding is performed before quantization, the variance of transmitted sequence depends 
on the behavior of the encoding matrix. In the syndrome-based DSC we transmit s = Hx. One can 



check that a s = a x [31 1. Unlike that, in the parity-based DSC, the variance of the parity samples is 
larger than that of the inputs. More precisely, in an (n, k) systematic DFT code, if c = G sys x, then 
°c = 7°z: where 7 = ^ tr (G^ s G sys ) > 1 1271. Since we can write c = [x \ p] T , we have 

= * (24) 



From 1 31 Theorem 7], we know that the smallest a p for a given DFT code is achieved when the parity 
samples, in the corresponding codewords, are located as "evenly" as possible. 

Considering the above arguments, one may expect the syndrome-based approach to perform better 
than the parity-based one, for a given code or fixed compression ratio. This is verified numerically in 
Section [Vl] The parity-based DSC, however, has other advantages. For example, by puncturing some 
parity samples rate-adaptive DSC, in the real field, is realized. Besides, it can be easily extended to 
distributed joint source-channel coding, as explained in the following section. 

V. Distributed Joint Source and Channel Coding 
The concept of lossy DSC and Wyner-Ziv coding using DFT codes was explained both for the syndrome 



and parity approaches in Section IV where syndrome or parity samples are quantized and transmitted over 
a noiseless channel. This implies separate source and channel coding. Although simple, the separation 
theorem is based on several assumptions, such as the source and channel coders not being constrained in 
terms of complexity and delay, which do not hold in many situations. It breaks down, for example, for 
non-ergodic channels and real-time communication. In such cases, it makes sense to integrate the design 
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Fig. 4. Joint source-channel coding (JSCC) with side information at the decoder based on DFT codes. Obviously, the scheme 
can be extended to distributed JSCC. 



of the source and channel coder systems, because joint source-channel coding (JSCC) can perform better 
given a fixed complexity and/or delay constraints. Likewise, distributed JSCC (DJSCC) has been shown 
to outperform separate distributed source and channel coding in some practical cases (34J. DJSCC has 
been addressed in |9j, |34|-|36|, using different binary codes. 

In this section, we extent the parity-based Wyner-Ziv coding of analog sources to the case where errors 
in the transmission are allowed. Thus, we introduce distributed JSSC of analog correlated sources in the 
analog domain. To do this, we use a single DFT code both to compress x and protect it against channel 
variations; this gives rise to a new framework for DJSCC, in which quantization is performed after doing 
JSCC in the analog domain. This scheme directly maps short source blocks into channel blocks, and thus 
it is well suited to low-delay coding. 



A. Coding and Compression 

To compress and protect x, the encoder generates the parity sequence p of n — k samples, with respect 
to a good systematic DFT code. The parity is then quantized and transmitted over a noisy channel, 
as shown in Fig. [4] To keep the dynamic range of parity samples as small as possible, we make use 
of optimal systematic DFT codes, proposed in pT| . This increases the efficiency of the system for a 
fixed number of quantization levels. Using an (n, k) DFT code a total compression ration of is 
achieved. Obviously, if n < 2k compression is possible. However, since there is little redundancy the 
end-to-end distortion could be high. Conversely, a code with n > 2k expands input sequence by adding 
soft redundancy to protect it in a noisy channel. 
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B. Decoding 

Let p = p + e c be the received parity vector which is distorted by quantization error q (p = p + q) 
and channel error e c . Also, let y = x + e v denote side information where e v represents the error due to 
the "virtual" correlation channel. The objective of the decoder is to estimate the input sequence from the 
received parity and side information. Although we only need to determine e v , effectively it is required 
to find both e v and e c . From an error correction point of view, this is equal to finding the error vector 
e = [e v e c ] T that affects the codeword [x p] T . Hence, to find the syndrome of error at the decoder, we 
append the parity p to the side information y and form z, a valid codeword perturbed by quantization 
and channel errors, 



X 













+ 


+ 




p 








Q 



G sys x + e + q'. 



Multiplying both sides by H, we obtain 



*z — ~i~ Sq' ) 



(25) 



(26) 



where s e = He and s q > = Hq' . Again, we use the GE model with q2 = in (|2]) to generate e. It should 
be emphasized that for q = 0, error vector can be determined exactly, as long as the number of errors 
is not greater than t. In practice, quantization is also involved, and we obtain only a distorted version of 
error syndrome. Knowing the syndrome of error, we use the error detection and localization algorithm, 



explained in Section III-B to find and correct error. 

VI. Simulation Results 

We evaluate the performance of the proposed systems using a Gauss-Markov source with mean 
zero, variance one, and correlation coefficient 0.9; the effective range of the input sequences is thus 
[—4,4]. The sources are binned using a (7,5) DFT code. The compressed vector, either syndrome or 
parity, is then quantized with a 6-bit uniform quantizer with step size A = 0.125 and transmitted 
over a noiseless communication media. The "virtual" correlation channel inserts t errors, generated by 
A/"(0, dg). The decoder detects, localizes, and decodes errors. We compare the MSE between transmitted 
and reconstructed codewords, to measurers end-to-end distortion. In all simulations, we use 10 5 input 



12 



We 



blocks for each channel-error-to-quantization-noise ratio (CEQNR), defined as o~ e la q and a\ 
vary the CEQNR and plot the resulting MSE. The results are presented in Fig. [5] and compared against 
the quantization error level in the existing lossy DSC methods. 
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Fig. 5. Reconstruction error in the syndrome and parity approaches, using a (7, 5) DFT code in Fig. 2, 3. For both schemes, 
the virtual correlation channel inserts one error at each input block. 



It can be observed that, in the syndrome approach the MSE is lower than the quantization error level 
in all CEQNRs. Analogously, in the parity approach, the MSE is less than that level except for a small 
range of CEQNR; further, it slightly improves when we use the adapted parity approach. Note that in 
lossy DSC using binary codes, the MSE can be equal to the quantization error level only if the bit 
error rate (BER) for the Slepian-Wolf coder is zero. This ideal case is not necessarily attainable even if 



capacity-approaching codes are used |25|. The performance of our systems considerably improves when 
CEQNR is very high. The improvement is due to a better error localization, since the higher the CEQNR, 
the better the error localization, as shown in Fig. [6] At very low CEQNRs, although error localization 
is poor, the MSE is still very low. This is because, compared to the quantization error, the errors are so 
small that the algorithm mostly does not detect (and thus localize) any error. Instead, it may occasionally 
localize and correct quantization errors. Note that, even if no errors are localized and corrected, the MSE 
is still very small as the errors are negligible at very low CEQNRs. Additionally, recall that the MSE 



is always reduced with a factor of R c = -, in an (n,k) DFT code |19|-|21|. It should be added that, 



for error detection, we use the algorithm proposed in Section III-B 1 ; for this purpose, we first find the 
threshold corresponding to pd = 90%; it leads to 9 = 0.0014. The corresponding plots are not included, 
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channel error to quantization noise ratio (dB) 



Fig. 6. Relative frequency of correct localization of correlation channel error in the syndrome and parity approaches, using 
a (7,5) DFT code. Since v = t = the coding theoretic and subspace error localizations give the same results; hence, 

we employ one of them in this set of plots. For the parity approach, we also plot the adapted error localization, introduced in 
Section [IV-B2| as well. 



for brevity; even so, similar plots are studied in the remainder of this section. 

In terms of compression, the efficiency of parity approach to the syndrome approach is R c = |, as 
discussed earlier in Section IV-C| Despite this, the performance of the parity approach is not as good 
as that of the syndrome approach. One reason is that in this simulation 1/5 of samples are corrupted in 
the parity approach while this figure is 1/7 for the syndrome approach. More importantly, the dynamic 
range of the parity samples, generated by ( fT8| ), could be much higher than that of the syndrome samples; 
this range depends on the position of the parity samples in the codewords. A wider range implies more 
quantization levels to achieve the same accuracy. For the (7, 5) code, in the best case we get a p = 2.17<r x 



since 7 m ; n = 2.0645 in ( |24| ). This is realized for a codeword pattern x x x — x x — , where "x's" and 
"— 's" represent data and parity samples, respectively. 

Next, we evaluate the performance of the JSCC with side information at the decoder, illustrated in 
Fig. [4] By using a systematic (10, 5) DFT code, we generate, quantize, and transmit parity samples over 



a noisy channel. Note that, for this code the best systematic code [27] achieves the lower bound in (24); 
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Fig. 7. Performance evaluation of joint source-channel coding with side information at the decoder, proposed in Fig. [4] for 
(10, 5) DFT code, (a) Histogram of A ma x(-R) for a quantized code. This is used to set a threshold for detection, (b) Probability 
of correct detection of errors. \(cj\ Probability of correct localization of errors, [(d)] The end-to-end distortion (MSE). 



i.e., it results in a p = a x . The correlation channel and transmission channel altogether insert up to t errors 
generated by J\F(Q, of). Simulation results are plotted in Fig. [7] First, based on Fig. 7(a) 



the threshold 



6 = 0.0065 is fixed for pd = 90%. Next, this is used to estimate v in Fig. 7(b) The estimated number is 



subsequently used to find the location of errors, both for the PGZ and subspace methods, in Fig. 7(c) 



Then, the output of Fig. 7(c) for the subspace method, is fed to the last step to find the magnitude 



It is worth mentioning that if the amplitude of errors is fixed, as assumed in [16J, the results improve considerably for both 
methods. For one thing, at the CEQNR of 20dB the probability of correct localization becomes 1. 
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of errors and correct them. Finally, in Fig. 7(d) the MSE is compared against the quantization error 
level, the ideal case in the lossy source coding based on binary codes. To put our results in perspective, 
we also calculate the MSE assuming perfect error localization; it gives 0, 6.5 x 10" & , and 1.8 x 10" 4 
respectively for 0, 1 , and 2 errors, in every CEQNR. This implies that there is still room to improve the 
MSE performance of the proposed system, given a better solution for the error localization. It also shows 
the performance gap between this DFT code and binary codes in the ideal case. 

It should be noted that capacity-approaching channel codes may introduce significant delay if one 
strives to approach the capacity of the channel with very a low probability of error. Therefore, those are 
out of the question for delay- sensitive systems. In that case, it would be best to use channel codes of low 
rate and focus on achieving a very low probability of error. The system we introduced is a low-delay 
system which works well with reasonably high-rate codes. Finally, by puncturing some parity samples, 
rate-adaptive schemes are realized for the proposed DJSCC and parity-based DSC. Rate-adaptive systems 



are popular in the transmission of non-ergodic data, like video [37]. 



VII. Conclusions 

We have introduced a new framework for the distributed lossy source coding, in general, and the Wyner- 
Ziv coding, in particular. The idea is to do binning before quantizing the continuous-valued signal, as 
opposed to the conventional approach where binning is done after quantization. By doing binning in 
the real field, the virtual correlation channel can be modeled more accurately and the quantization error 
can be compensated for when there is no error. In the new paradigm, Wyner-Ziv coding is realized by 
cascading a Slepian-Wolf encoder with a quantizer. We employ real BCH-DFT codes to do the Slepian- 
Wolf in the real field. At the decoder, by introducing both syndrome-based and parity-based systems, we 
adapt the PGZ decoding algorithm of channel coding to DSC. The extension of the parity-based Wyner- 
Ziv coding to joint source-channel coding with side information at the decoder is straightforward. This 
scheme directly maps short source blocks into channel blocks, and thus it is appropriate for low-delay 
coding. From simulation results, we conclude that our systems can improve the reconstruction error even 
using short codes, so they can become viable in real-world scenarios where low-delay communication is 
required. 

We have also adapted the subspace error localization in this context and improved it for the parity-based 
scheme. This reasonably improves the error localization and leads to a better mean-squared reconstruction 
error. We should point out that, a more accurate algorithm for error localization is a key to further improve 
the reconstruction error. Particularly, error localization becomes more challenging when codeword length 
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increases, as the roots of error locator polynomial, which are over the unit circle, get closer. 

VIII. Appendix 

A. Proof of Lemma [7] 

Proof: The proof is straightforward; we show this for odd k and leave the other case to the reader. 
Since a = f^i^] and d = n — k, using ([8]), for odd k we can write 

s d—m+l — 7= / , e i„si-p 

p=i 

l v -fc + i _ m 

v p=i 

Note that X™ = l, for any p. ■ 
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